Multi-center validation of an artificial intelligence system for detection of COVID-19 on chest radiographs in symptomatic patients.

OBJECTIVES: While chest radiograph (CXR) is the first-line imaging investigation in patients with respiratory symptoms, differentiating COVID-19 from other respiratory infections on CXR remains challenging. We developed and validated an AI system for COVID-19 detection on presenting CXR. METHODS: A deep learning model (RadGenX), trained on 168,850 CXRs, was validated on a large international test set of presenting CXRs of symptomatic patients from 9 study sites (US, Italy, and Hong Kong SAR) and 2 public datasets from the US and Europe. Performance was measured by area under the receiver operator characteristic curve (AUC). Bootstrapped simulations were performed to assess performance across a range of potential COVID-19 disease prevalence values (3.33 to 33.3%). Comparison against international radiologists was performed on an independent test set of 852 cases. RESULTS: RadGenX achieved an AUC of 0.89 on 4-fold cross-validation and an AUC of 0.79 (95%CI 0.78-0.80) on an independent test cohort of 5,894 patients. Delong's test showed statistical differences in model performance across patients from different regions (p < 0.01), disease severity (p < 0.001), gender (p < 0.001), and age (p = 0.03). Prevalence simulations showed the negative predictive value increases from 86.1% at 33.3% prevalence, to greater than 98.5% at any prevalence below 4.5%. Compared with radiologists, McNemar's test showed the model has higher sensitivity (p < 0.001) but lower specificity (p < 0.001). CONCLUSION: An AI model that predicts COVID-19 infection on CXR in symptomatic patients was validated on a large international cohort providing valuable context on testing and performance expectations for AI systems that perform COVID-19 prediction on CXR. KEY POINTS: • An AI model developed using CXRs to detect COVID-19 was validated in a large multi-center cohort of 5,894 patients from 9 prospectively recruited sites and 2 public datasets. • Differences in AI model performance were seen across region, disease severity, gender, and age. • Prevalence simulations on the international test set demonstrate the model's NPV is greater than 98.5% at any prevalence below 4.5%.

Breath-holding physiology, radiological severity and adverse outcomes in COVID-19 patients: A prospective validation study.

BACKGROUND AND OBJECTIVE: COVID-19 remains a major cause of respiratory failure, and means to identify future deterioration is needed. We recently developed a prediction score based on breath-holding manoeuvres (desaturation and maximal duration) to predict incident adverse COVID-19 outcomes. Here we prospectively validated our breath-holding prediction score in COVID-19 patients, and assessed associations with radiological scores of pulmonary involvement. METHODS: Hospitalized COVID-19 patients (N = 110, three recruitment centres) performed breath-holds at admission to provide a prediction score (Messineo et al.) based on mean desaturation (20-s breath-holds) and maximal breath-hold duration, plus baseline saturation, body mass index and cardiovascular disease. Odds ratios for incident adverse outcomes (composite of bi-level ventilatory support, ICU admission and death) were described for patients with versus without elevated scores (>0). Regression examined associations with chest x-ray (Brixia score) and computed tomography (CT; 3D-software quantification). Additional comparisons were made with the previously-validated '4C-score'. RESULTS: Elevated prediction score was associated with adverse COVID-19 outcomes (N = 12/110), OR[95%CI] = 4.54[1.17-17.83], p = 0.030 (positive predictive value = 9/48, negative predictive value = 59/62). Results were diminished with removal of mean desaturation from the prediction score (OR = 3.30[0.93-11.72]). The prediction score rose linearly with Brixia score (ß[95%CI] = 0.13[0.02-0.23], p = 0.026, N = 103) and CT-based quantification (ß = 1.02[0.39-1.65], p = 0.002, N = 45). Mean desaturation was also associated with both radiological assessment. Elevated 4C-scores (≥high-risk category) had a weaker association with adverse outcomes (OR = 2.44[0.62-9.56]). CONCLUSION: An elevated breath-holding prediction score is associated with almost five-fold increased adverse COVID-19 outcome risk, and with pulmonary deficits observed in chest imaging. Breath-holding may identify COVID-19 patients at risk of future respiratory failure.

Chest X-ray versus chest computed tomography for outcome prediction in hospitalized patients with COVID-19.

The purpose of this study was to compare the prognostic value of chest X-ray (CXR) and chest computed tomography (CT) in a group of hospitalized patients with COVID-19. For this study, we retrospectively selected a cohort of 106 hospitalized patients with COVID-19 who underwent both CXR and chest CT at admission. For each patient, the pulmonary involvement was ranked by applying the Brixia score for CXR and the percentage of well-aerated lung (WAL) for CT. The Brixia score was assigned at admission (A-Brixia score) and during hospitalization. During hospitalization, only the highest score (H-Brixia score) was considered. At admission, the percentage of WAL (A-CT%WAL) was quantified using a dedicated software. On logistic regression analyses, H-Brixia score was the most effective radiological marker for predicting in-hospital mortality and invasive mechanical ventilation. Additionally, A-CT%WAL did not provide substantial advantages in the risk stratification of hospitalized patients with COVID-19 compared to A-Brixia score.

Lombardy, Northern Italy: COVID-19 second wave less severe and deadly than the first? A preliminary investigation.

BACKGROUND: The much-heralded second wave of coronavirus disease (COVID-19) has arrived in Italy. Right now, one of the main questions about COVID-19 is whether the second wave is less severe and deadly than the first wave. In order to answer this challenging question, we decided to evaluate the chest X-ray (CXR) severity of COVID-19 pneumonia, the mechanical ventilation (MV) use, the patient outcome, and certain clinical/laboratory data during the second wave and compare them with those of the first wave. METHODS: During the two COVID-19 waves two independent groups of hospitalised patients were selected. The first group consisted of the first 100 COVID-19 patients admitted to our hospital during the first wave. The second group consisted of another 100 consecutive COVID-19 patients admitted to our hospital during the second wave. We enlisted only Caucasian male patients over the age of fifty for whom the final outcome was available. For each patient, the CXR severity of COVID-19 pneumonia, the MV use, the patient outcome, comorbidities, corticosteroid use, and C-reactive protein (CRP) levels were considered. Nonparametric statistical tests were used to compare the data obtained from the two waves. RESULTS: The CXR severity of COVID-19 pneumonia, the in-hospital mortality, and CRP levels were significantly higher in the first wave than in the second wave (p ≤ .041). Although not statistically significant, the frequency of MV use was higher in the first wave. CONCLUSIONS: This preliminary investigation seems to confirm that the COVID-19 second wave is less severe and deadly than the first wave.

Chest X-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: A study of 302 patients from Italy.

OBJECTIVES: This study aimed to assess the usefulness of a new chest X-ray scoring system - the Brixia score - to predict the risk of in-hospital mortality in hospitalized patients with coronavirus disease 2019 (COVID-19). METHODS: Between March 4, 2020 and March 24, 2020, all CXR reports including the Brixia score were retrieved. We enrolled only hospitalized Caucasian patients with COVID-19 for whom the final outcome was available. For each patient, age, sex, underlying comorbidities, immunosuppressive therapies, and the CXR report containing the highest score were considered for analysis. These independent variables were analyzed using a multivariable logistic regression model to extract the predictive factors for in-hospital mortality. RESULTS: 302 Caucasian patients who were hospitalized for COVID-19 were enrolled. In the multivariable logistic regression model, only Brixia score, patient age, and conditions that induced immunosuppression were the significant predictive factors for in-hospital mortality. According to receiver operating characteristic curve analyses, the optimal cutoff values for Brixia score and patient age were 8 points and 71 years, respectively. Three different models that included the Brixia score showed excellent predictive power. CONCLUSIONS: Patients with a high Brixia score and at least one other predictive factor had the highest risk of in-hospital death.

Radiographic severity index in COVID-19 pneumonia: relationship to age and sex in 783 Italian patients.

PURPOSE: To improve the risk stratification of patients infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), an experimental chest X-ray (CXR) scoring system for quantifying lung abnormalities was introduced in our Diagnostic Imaging Department. The purpose of this study was to retrospectively evaluate correlations between the CXR score and the age or sex of Italian patients infected with SARS-CoV-2. MATERIALS AND METHODS: Between March 4, 2020, and March 18, 2020, all CXR reports containing the new scoring system were retrieved. Only hospitalized patients with SARS-CoV-2 infection were enrolled. For each patient, age, sex, and the CXR report containing the highest score were considered for the analysis. Patients were also divided into seven groups according to age. Nonparametric statistical tests were used to examine the relationship between the severity of lung disease and the age or sex. RESULTS: 783 Italian patients (532 males and 251 females) with SARS-CoV-2 infection were enrolled. The CXR score was significantly higher in males than in females only in groups aged 50 to 79 years. A significant correlation was observed between the CXR score and age in both males and females. Males aged 50 years or older and females aged 80 years or older with coronavirus disease 2019 showed the highest CXR score (median ≥ 8). CONCLUSIONS: Males aged 50 years or older and females aged 80 years or older showed the highest risk of developing severe lung disease. Our results may help to identify the highest-risk patients and those who require specific treatment strategies.